System and method for camera calibration

ABSTRACT

A dermatoscope or endoscope inspection device which can be calibrated at high accuracy to be able to focus at a certain depth, e.g. below the top surface of the skin. A calibration pattern is provided or can be located at a reference viewing surface of an inspection device such as a dermatoscope or endoscope. It is important for a dermatoscope to know with the best accuracy available at what absolute depth below the top of the skin the device is focused. The dermatoscope or endoscope inspection device includes focusing means and by knowing a relationship between a digital driving level which shifts the focus position of the focusing means and the corresponding absolute change in focus depth, it is possible to know how deep the device is focused in absolute terms below the top of the skin.

FIELD OF THE INVENTION

The invention relates to the field of diagnostic imaging, and to aninspection device and how to calibrate a camera of an inspection devicefor diagnostic imaging.

BACKGROUND OF THE INVENTION

A dermatoscope can be used for diagnostic imaging of skin. Today mostdermatoscopes are still anologue devices, meaning they comprise amagnifying glass with lighting around it. Dermatoscopes do not only lookat the top surface of the skin, but they image at different depths inthe skin, down to about 3 mm or more. The dermatologist can adjust thefocus such that the focal point is set to a “desired depth”, and this isdecided by the dermatologist based on experience. As the focus ischanged, specific skin structures will come into focus and other oneswill go out of focus. The expert dermatologist recognizes thesestructures and knows continuously where he/she is when browsing the skin(e.g. at the top, the epidermis, the dermis, . . . ). For non-experts,however, this is very difficult and most often non-experts don't knowexactly at what depth the device is focused.

The diagnosis of certain types of skin cancer is not easy and analoguedevices do not record the images seen by the doctor when making thediagnosis. This results in a conservative approach of making morebiopsies than would be required if a better method of diagnosis wereavailable.

FIG. 1 shows a schematic representation of different skin layers on they axis, and the penetration depth (in mm) for light of differentwavelengths (in nm) on the x axis. In fact, the longer the wavelength,the deeper it penetrates into the skin tissue and the deeper is thefocus corresponding to a certain wavelength. In each layer, there arecharacteristic structures present, thus there will be “sharp” structuresvisible from the surface and down to more than 3 mm depth. Thus,conventional autofocus algorithms, e.g. such used in smartphones, willnot work since such algorithms rely on the image being sharp once thefocal point is set correctly.

Recently digital dermatoscopes have been introduced. These devices arethe digital equivalent of the regular analogue dermatoscope. Today,digital dermatoscopes offer mostly the same functionality as analoguedermatoscopes; but they can take an image of the skin and thedermatologist can (manually) control the focus depth.

With the emergence of digital dermatoscopes it becomes possible todecouple the “imaging” of a skin lesion from the “reading/diagnosing” ofit. General Practitioners (GP's) or nurses could e.g. image skin lesionsand send the images to dermatologists for diagnosis.

However, this requires that images are acquired at the correct focusdepth (or rather at a multitude of correct focus depths) and that it isknown which focus depth is associated with which image.

Moreover, it is crucial to know with a very high accuracy the depth ofthe lesion. In fact, in the case of melanoma for example, the depth ofthe lesion provides an estimate of the chances of survival of thepatient after 5 years. The depth of the melanoma is one of the mostcrucial parameters to estimate in order to obtain an accurate diagnosis.Therefore, the focus depth needs to be perfectly controlled throughoutthe lifetime of the device.

US20160282598A1 (Canon 3D) discloses how to calibrate a microscope fordoing e.g. 3D analysis, by using a calibration target with a testpattern. The calibration target is physically positioned on a movablestage below the optical system.

US20160058288A1 (Mela, dermatoscope) discloses three-dimensional imagingof tissue, using “the known camera calibration of the system” ascalibration reference.

SUMMARY OF THE INVENTION

With respect to inspection devices for medical applications such asdermatological inspection devices or endoscopes there is a need to:

-   -   Provide a method for automatically controlling focus depth when        imaging targets that have structures throughout its volume,        since automatic focus algorithms as they exist today are not        suitable.    -   Be able to associate digital driving values that control focus        to an absolute value of the focus depth, and to associate a        range of digital driving values that control focus to a range of        absolute values of the focus depth.    -   Linking acquired images to specific absolute focus depth values.

In an aspect, the present invention provides methods to calibrate athigh accuracy the focus position at the top surface of the skin. Forthis reason, in a preferred embodiment, a calibration pattern isprovided or can be located at a reference viewing surface of aninspection device such as a dermatoscope or endoscope. The referenceviewing surface can be on the lower surface of the front glass of theinspection device, which during use is in contact with the skin of apatient or is inserted into a body cavity at the end of an endoscope. Itis also advantageous for a dermatoscope to know with the best accuracyavailable at what absolute depth below the top of the skin the device isfocused. This can be achieved by embodiments of the present invention byproviding focusing means and by knowing a relationship between a digitaldriving level which shifts the focus position of the focusing means andthe corresponding absolute change in focus depth. From this it ispossible to know how deep the device is focused in absolute terms belowthe top of the skin.

Similar considerations apply to an endoscope.

More than one calibration patterns can be used. By using two patterns atdifferent depths where the difference in depth between these twopatterns is known, the relationship between difference in depth anddriving levels of the focusing means can be established by focusing onthese two patterns and determining the corresponding difference indigital driving value that shifts the focus position. The relationshipbetween the absolute difference in focus depth and the correspondingchange in digital driving value is then established. Such a relationshipis a calibration.

To allow repeated calibration a reference distance is required that doesnot change over the lifetime of the inspection device. Embodiments ofthe present invention select this distance to be a known fixed depthrelationship between the calibration pattern and the front side of theglass which during inspection will be applied directly to the skin of apatient.

An advantage for locating the calibration pattern at a known distance tothe front glass reference viewing surface is that it allowsautofocusing. A typical existing auto focus algorithm changes focusiteratively up to the point where maximum sharpness of the image isobtained. It is assumed that the focus is then set correctly. This workswith natural scenes (photographs) where the picture is not sharp whenthe focus is wrong.

But this does not work in with diagnostic images of the skin as the skincontains well defined structures at different depths. Therefore,changing the focus results in several depths where sharpness isobtained. And moreover, if autofocus were to be performed, the depth atwhich the focus is obtained would not be a reliable value.

Providing a calibration a pattern on the front glass reference viewingsurface, which is preferably the surface to be in contact with the skin,means that the focus is therefore on the top of the skin and not withinit since the glass is pressed on the skin.

Embodiments of the present invention may make use of extensions to theviewing end of the inspection device. In such a case the referenceviewing surface is that surface which comes in contact with the skinwhich may not be front glass. The extensions can be tips of differentlengths. Thus, a main goal for calibration can be to be able to focus onthe end of the extension, and more precisely the surface to be incontact with the skin.

A further problem with focusing devices using a mechanical system tochange focus having moving parts is backlash. Backlash can result indifferent focus positions depending upon the direction of travel of thefocusing device. There are known methods of avoiding backlash but theycan increase the bulk of the inspection unit. Embodiments of presentdevice avoid backlash by the use of deformable lenses.

In one aspect of the invention calibrated inspection unit is providedfor direct application to the skin of a patient or for examining theinterior of a hollow organ or cavity of the body of a patient, saidcalibrated inspection unit comprising an optical array of elements in anoptical path comprising:

-   -   at least one light source,    -   an image capturing device having a field of view,    -   an imaging lens having a radius of curvature and a focal length        defining a position of a focus point along the optical path,    -   focusing means for changing the focus position along the optical        path as a function of adjustment values, and    -   a reference viewing surface through which images are captured        for the image capturing device, characterized by    -   a calibration pattern for locating in a fixed position with        respect to the reference viewing surface and in the field of        view of the image capturing device, and    -   a calibration means for defining a relationship between first        adjustment values of the focusing means and the positions of        focus points along the optical path, including a second        adjustment value for a focus position at the fixed position of        the calibration pattern.

The calibration means can be a relationship that is pre-calculated andstored or the calibration means can generate the relationship at anytime, e.g. starting from the stored second adjustment value being forexample the digital value corresponding to a reference position. Then a“distance step per change in digital drive” can be determined tocomplete the calibration.

The calibration pattern can not only be used for focus calibration, butcan also be used for absolute color calibration. A single calibrationpattern can be used or a combination of multiple calibration patterns,the calibration patterns having different functions. Accordingly, thecalibration pattern can comprise a color calibration chart. The colorcalibration chart can be used for absolute color calibration. This hasthe advantage that it is possible to include absolute color calibrationand correct possible drift of the light sources.

The present invention provides means suitable for referring to anabsolute reference point of calibration of an image capturing device.And when the unit is investigating objects having a multiple of possiblefocus points at different depths, the unit can be instructed to focusthe light at a required absolute depth.

The calibration using the calibration pattern or patterns can alsoprovide a means of correcting a predefined or pre-calibratedrelationship of a focus position along the optical path as a function ofadjustment values at a later date, e.g. by the practicing doctor.

The at least one light source can be centred on a first wavelength andhave a first spectral bandwidth.

This makes it possible to work with light of a certain colour orwavelength range, having desired properties, for example penetrationdepth into skin which is wavelength dependent.

The at least one light source may comprise a plurality of light sources,each light source being centred on a different wavelength and having aspectral bandwidth, and the first adjustment values of the focusingmeans and the positions of the focus points along the optical path canbe different for each wavelength.

This makes it possible to have the inspection unit operating withcombinations of several colours or wavelength bands and hence combinetheir properties. Additionally, the light sources can be independentlycontrolled, which enables a customized calibration and operation.

There can be a second calibration pattern provided for locating in asecond fixed known position with respect to said reference position andin the field of view of the image capturing device, and a stored thirdadjustment value for a second focus position at the second fixed knownposition of the second calibration pattern. This makes it possible torelate to a second absolute focus point which can be located at a knownposition from the first focus point, so that adjustment values can berelated to an absolute depth. Additionally, two or more calibrationpatterns can provide an estimate of a step size along the optical axisof the optical path related to a change of an adjustment value. Therelationship between a focus position along the optical axis and theadjustment values might shift due to aging. With two or more patterns atdifferent depths, it is possible to identify and hence to correct forsuch a drift.

In addition there can be further calibration patterns ate points locatedbetween positions of the first and second calibration pattern. This canprovide higher calibration accuracy. An example is first and secondcalibration patterns on each side of the front glass with extracalibration patterns embedded in the glass at intermediate positions.

Accordingly, there can be a front plate provided in the exit pupil ofthe optical array, and wherein the calibration pattern can be providedon at least one of the two surfaces of the front plate and/or inside thefront plate.

Thus, the calibration pattern may be place on, or inside, the frontglass of the inspection unit.

In another aspect of the invention, the calibration pattern can beplaced on the skin of a patient as a tattoo or a stamp. This makes itpossible to have calibration points on the actual object to beinvestigated.

A calibration pattern can be placed on a substrate being thinner thanthe front glass, which is positioned on top of the front glass or insidethe housing, or the calibration pattern is put directly on a part of thehousing.

Alternatively, the calibration pattern can be put on a substrate or filmthat can be added between the front glass and the object to beinvestigated, e.g. the skin. Alternatively, the substrate with a patterncan be put inside the housing of the inspection unit. Alternatively, thecalibration pattern can be put directly onto a part of the housing. Inall cases the substrate with pattern is put within the field of view ofthe imaging device.

In another aspect of the invention, the focusing means can be providedby any of: The imaging lens which is a liquid lens or the imagecapturing device which can be configured to be translated along theoptical axis, or the imaging lens which can be configured to betranslated along the optical axis. Additionally, the first and secondadjustment values the above configurations can be driving voltages. Whenthe imaging lens is a liquid lens, the focal length of the lens in anadditional piece keeps the camera lens operating in a range with highsensitivity.

Thus, the inspection unit may be implemented by using a high precisionliquid lens, and/or a moving sensor, and/or a moving lens, for which thefocus position along the optical axis can be adjusted by changing adriving voltage.

In another aspect of the invention the focusing means can comprise meansfor calculating the modulation transfer function of the optical array.

This makes it possible to obtain information on for example theresolution of the camera.

In another aspect of the invention, the calibration pattern can be athree-dimensional pattern defining a plurality of fixed known positionsfrom the reference position when said three-dimensional pattern isinstalled on the inspection device at the fixed distance to thereference viewing surface. The focusing means further can have aplurality of adjustment values for focus positions at a plurality offixed known positions of the three-dimensional calibration pattern, e.g.when translated in the x, y plane, whereby the calibration patterns aremade in such a way that the position in the x, y plane can be accuratelydetermined. Additionally, a three-dimensional calibration pattern can beengraved, etched, milled or printed within a calibration substrate.Additionally, the calibration pattern can comprise parallel lines whichare in a plane which form an angle with the plane of the imageacquisition device. Additionally, the calibration pattern can comprise apattern which is in a plane parallel to the plane of the sensor.Additionally, the distances between patterns can be correlated to theirdistance from the surface of the substrate.

This aspect of the invention makes it possible to calibrate towardsabsolute points outside or beyond the inspection unit. The calibrationpattern can be a phantom of human skin, which could be used to check theaccuracy of the calibration. The calibration pattern can also be formedby part of the housing of the unit, e.g. edges of the housing coulddetected by edge detecting methods.

In another aspect of the invention, the calibrated inspection unit canbe configured to operate with a second piece providing a second exitpupil. Additionally, a second front glass is provided at the second exitpupil and at least one calibration pattern is provided at a knowndistance from the second front glass. Additionally, the second piece cancomprise a lens whose focal length relates to the length of the piece.Additionally, the imaging lens can be a liquid lens and the focal lengthof the lens in the additional piece keeps the camera lens operating in arange with high sensitivity, for example a small focal depth step size.

This makes it possible to provide a shape of the housing of theinspection unit that has a better suitable geometry, for example, a moreelongated narrow shape that can reach the skin in narrow regions likebetween fingers. If e.g. a liquid lens is used it is desirable to keepit working in its most sensitive range where fine tuning is possible. Anadditional lens in the second piece can alter the focus length to bettersuit the liquid lens. Hence when the imaging lens is a liquid lens, thefocal length of the lens in an additional piece keeps the camera lensoperating in a range with high sensitivity.

In another aspect of the invention, there can be calibration informationput in optical codes such as a barcodes or QR (Quick Response) codesnext to the calibration pattern.

The optical code such as a barcode or QR code can store relevantinformation on the calibration pattern and/or information related to theuser, for example a customized calibration procedure.

A means can be provided to detect if the front surface is in contactwith an external object, such as the skin, e.g. by use of a touch screenin the front surface. This could be used to known when absolute focussetting is preferred and when an auto focus algorithm is preferred.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows penetration depths in the skin for different wavelengths.

FIG. 2 shows the fundamental parts of a dermatoscope.

FIG. 3 a ) shows an example of calibration patterns and FIG. 3 b ) showsan embodiment of the present invention comprising a front glass withcalibration pattern.

FIG. 4 shows an embodiment of the present invention comprising a frontglass having two calibration patterns at a distance from each other.

FIG. 5 shows an embodiment of the present invention comprising anextension piece.

FIG. 6 shows the characteristics of a liquid lens.

FIG. 7 shows an embodiment of the present invention comprising athree-dimensional calibration pattern.

FIG. 8 shows a flow chart of an embodiment of the present invention.

FIG. 9 shows an example of a relative focus function.

FIG. 10 shows an embodiment of the present invention comprising acalibration pattern on a charging station.

FIG. 11 shows a schematic side view of a dermatoscope according toembodiments of the present invention.

FIG. 12 shows the spectral sensitivity of a sensor according to thepresent invention.

FIG. 13 shows the normalized spectral power distributions of lightsources used with embodiments of the present invention.

FIG. 14 the Skin reflectance in the wavelength regions of UVB (280-320nm), UVA (320-400 nm) and visible (400-700 nm) for three differentconcentrations of melanosomes in the epidermis corresponding to skintypes II, III and IV, respectively. This image is from “The optics ofhuman skin: Aspects important for human health”, by Kristian PaghNielsen, Lu Zhao, Jakob J Stamnes, Johan Moan in Solar Radiation andHuman Health, Oslo: The Norwegian Academy of Science and Letters, 2008.

FIG. 15 shows a block diagram illustrating a method according to thepresent invention.

FIG. 16 shows a close up of FIG. 13 in the region of the overlap betweenblue and green light sources.

FIG. 17 shows how calibration curves representing different drive levelsfor focus positions can change with time due to various environmentaleffects.

DETAILED DESCRIPTION OF THE INVENTION

The present invention will be described with respect to particularembodiments and with reference to certain drawings but the invention isnot limited thereto but only by the claims. The drawings described areonly schematic and are non-limiting. In the drawings, the size of someof the elements may be exaggerated and not drawn on scale forillustrative purposes. Where the term “comprising” is used in thepresent description and claims, it does not exclude other elements orsteps. Furthermore, the terms first, second, third and the like in thedescription and in the claims, are used for distinguishing betweensimilar elements and not necessarily for describing a sequential orchronological order. It is to be understood that the terms so used areinterchangeable under appropriate circumstances and that the embodimentsof the invention described herein are capable of operation in othersequences than described or illustrated herein.

The present invention relates to optical systems with variable opticalfocus and relate to solving the problem of variability incharacteristics. Focus in such a system can be controlled by means ofdigital values or driving levels. These digital values are applied to anactuator which can either mechanically change the position of a lens orlenses to change the focus, or in the case of a liquid lens (for examplefrom Varioptic®) applies voltage to change the curvature of a liquidlens and therefore the focus. The relationship between these digitalvalues and the exact focus depth can suffer from variability, e.g.caused by mechanical tolerances, liquid lenses being sensitive totemperature effects, . . . . Embodiments of the present invention makesure that a digital drive value will always be associated with aspecific focus depth. Also the optical system of an inspection deviceaccording to embodiments of the present invention can be calibrated atmanufacturing, and corrections can be made later to correct for theaccuracy of the focusing system being influenced by temperature changeswhich could change the focus depth. In critical applications such asdermatology these changes in focus depth can be of the same magnitude oreven larger than the accuracy required in focusing (e.g. 100-120 μm) theinstrument for an accurate diagnosis.

In accordance with embodiments of the present invention adjustmentvalues for focusing means which correspond to a given focus positionalong the optical axis, the right hand cross in FIG. 17 combined withfocus positions along the optical axis as a function of adjustmentvalues as shown in the lower solid line compensation or correct can beused (using a pre-determined potentially non-linear transformation) ofthe focus position along the optical axis as a function of adjustmentvalues as can be seen from the upper solid line.

This can be done with only one calibration point being remeasured whenthe reference adjustment value corresponding to the given focus positionalong the optical axis changes (cross on the left), due to for exampleaging, environmental conditions, mechanical change (due to shock,vibration . . . ).

FIG. 2 illustrates a basic configuration of a digital dermatoscopeaccording to embodiments of the present invention. It comprises ahousing 10 (of arbitrary shape), an image capturing device such as acamera sensor 16, a camera lens 11, and a front glass 14 which forms areference viewing surface. At least one light source is provided, or aplurality of light sources 17 are provided, but the dermatoscope can inprinciple function with ambient light. The lens can for example be avariable focus liquid lens such as the “Caspian S-25H0-096” fromVarioptic®. The sensor board can for example be “LI-IMX274-MIPI-M12”from Leopard Imaging Inc. Internal light sources can be implemented withfor example the Luxeon® C or Z LED series from Lumileds®.

It is important to note that although the term front glass is usedthroughout the application, it does not need to be glass, it can be anytransparent substrate, for example a transparent plastic substrate, etc.Also means can be provided to detect when the viewing surface touchesthe skin, the viewing surface can be a touch screen.

FIG. 11 shows a schematic side view of a dermatoscope 110 according toembodiments of the present invention. Embodiments of the presentinvention may also be used at the end of an endoscope.

The dermatoscope 110 comprises advantageously a plurality of lightsources 1110 having different spectral bands, wherein the focusassociated to each spectral band is at a different depth in human oranimal skin, as shown on FIG. 1 . Thus, the dermatoscope 110 usesmultispectral imaging. The dermatoscope 110 sequentially (or inembodiments of the present invention simultaneously) illuminates aregion of interest of the skin with light sources of different spectralbands. Each time an image is acquired a multispectral scan is made.

In an embodiment of the present invention, the light sources can be LEDslight sources.

The dermatoscope 110 further comprises an image acquisition device 1120,which can be a CCD or a camera sensor, CMOS sensor or a line scanner.Thus, the term “sensor” refers to any suitable image acquisition device1120.

The image acquisition device, e.g. sensor used in an embodiment of thepresent invention provides a resolution at the surface of the skin ofabout 6.6 μm. The field of view of the device shown on FIG. 11 is27.5×16 mm, at the surface of the skin. At deeper depths, the field ofview is slightly increased. The field of view of the device ispreferably selected so that it can view common types of skin defects,lesions etc.

The spectral sensitivity of the image acquisition device, e.g. sensorhas to match the spectral bandwidth of the light sources. Greyscalesensors can be used for the present application, however, although colorsensors may also be used for the present invention.

The spectral sensitivity of a Sony sensor which can be used withembodiments of the present invention is shown on FIG. 12 . The solidline shows the Sony IMX214 and the dashed line shows the Sony IMX135.

In an example of an embodiment of the present invention, a plurality oflight sources such as LEDs are arranged in two arrays, i.e. LED arrays,each array, i.e.LED array comprising seven light sources, e.g. LEDs.

In a preferred embodiment of the present invention, the light sourcesare arranged in a ring. Two light sources of the same type (i.e. colorand optionally polarization) can be placed on the opposite side of thering from each other. In an embodiment according to the presentinvention, seven different types of light sources, e.g. LED lightsources having different spectral bands are arranged in a ring.

The following spectral bands can be selected:

-   -   1. White unpolarized    -   2. Blue unpolarized    -   3. White polarized    -   4. Blue polarized    -   5. Green polarized    -   6. Deep red polarized    -   7. Far red polarized

To generate polarized light, a polarizer 1125 may be inserted in thedevice as shown on FIG. 11 .

Further, a cross polarizer 1140 (the combination of the two polarizerswhich are crossed with respect to each other) can be used in front ofthe sensor 1120 of the image acquisition device. Such a cross polarizercan filter out parasitic reflections due to light scattering. The imageacquisition device is further provided on a PCB sensor board 1145.

An LED PCB board 1130 is provided in a vicinity of the LED array 1110 asshown on FIG. 11 .

Optionally, a fastening means such as magnets can be placed near thefront edges of the housing (in a proximity of the front plate) which canbe used to fix a cone extension piece, described below.

All these components are arranged in a housing 1150. The housing can becone shaped, or rectangular, cylindrical, etc.

FIG. 13 shows the normalized spectral power distributions of LED lightsources used with embodiments of the present invention. The LEDs may beacquired from the supplier “Lumiled” and the series is called “Luxeon Z”for blue, green, deep red and white, and “Luxeon C” for far red.Different LED's are used for the different colors. FIG. 16 shows a closeup of FIG. 13 in the region of the overlap between the blue and greenlight sources.

In the example shown on FIG. 11 , the optical centre of each LED 1110 ispositioned on a circle having a radius of 7.5 mm, in exception to thefar-red polarized LED for which the circle has a radius of 7.9 mm.

The centre of the LED ring corresponds to the image acquisition deviceoptical axis and has an opening for the lens, for example a liquid lens.The radius of the opening can be 5 to 15 6.5 mm for example.

The front glass 1105 of the device of the present invention can comprisean anti-reflection coating on the front glass, provided on one or bothsides of the front glass. This coating preferably provides as littlefiltering as possible, and no other filters need to be used. The frontglass has an upper surface which is inside the device and a lowersurface which, during use, can be in direct contact with the skin of apatient. The thickness of the front glass is preferably in the range of1 to 2 mm.

In order to calibrate the focus position, digital driving levels offocusing means are determined to generate a calibration, depending onthe type of dermatoscope used. In some embodiments, the focusing meanscan be provided by the image acquisition device which can be translatedalong the optical axis of the optical path, and therefore its positionwith respect to the focus point needs to be calibrated. In otherembodiments, the focusing means can be provided by changing the positionof an imaging lens, the imaging lens being able to move along theoptical axis of the optical path. In other embodiments, the focusingmeans can be provided by a liquid lens wherein it is the change incurvature of the -liquid lens which can be modified, for example a lensfrom Varioptic®, wherein the voltage changes the curvature of a liquidlens and therefore the focus position. In all these embodiments, thefocus is controlled by means of digital values or driving levels. Whenthe imaging lens is a liquid lens, the focal length of the lens in anadditional piece keeps the camera lens operating in a range with highsensitivity.

The device shown on FIG. 11 comprises a liquid lens 1115 whose distanceto the front glass is 40 mm. This distance is calculated so that theliquid lens is in the flat position, and thus the optical system haslimited aberrations and distortions and has the smallest focus depthsize, and provides the best optical performance. The precision of thefocus depth of the optical system with the liquid lens is ofapproximately 120 μm.

The focus depth as a function of the digital driving value for thefocusing means is often not linear throughout the entire range ofmovement of the moving part of the focusing means at a given wavelength.Therefore, the present invention provides different types ofcalibrations: a local and quick calibration which can be calibratediteratively each time by evaluating the sharpness and changing the focusso that a sharpness metric is maximized, a broad calibration which spansa broader range of focus positions along the optical path, and whichallows to detect if the focusing means is outside of a linear range.

The accuracy of both types of calibrations is of the same order. Thebroad calibration can be performed using an additional substrate, whichenables determination of the relationship between the digital drivinglevel and the focus on the pattern in the field of view, of which thefocus depth is known. For a further focus positions the focusing meansis used to determine the digital driving levels for different focusdepths. This can be done using a calibration target which has patternsat different focus depths. During this broad calibration, therelationship between the focus depth for the fixed pattern and a rangeof focus depths corresponding to depths deeper in the skin, i.e. beyondthe front of the device is established. This type of calibration can beperformed in the factory but also later, by a technician on site, forexample or can be implemented within a docking station, while chargingthe device.

The local and quick calibration performs the first part of the broadcalibration, but is used to take into account the external conditions ofoperation of the image acquisition, and thus takes into account thetemperature, etc. Based on this self-calibration, the calibration usingan external object is corrected. This way the relationship of thedigital driving levels to focus positions at different depths is knownmore accurately, even with the absence of the external calibrationtarget (broad calibration).

The range of the broad calibration can be optionally limited to thefirst millimeters of focus distance, in order to capture skin lesions.Although the optical system is capable of going above this range, theabsolute focus depth is calibrated in this range where a skin lesion islikely to occur. For an endoscope a larger range can be used.

This broad calibration can be performed in the factory but in order toensure the reliability of the device throughout its lifetime it ispreferable to provide means for performing this broad calibrationwhenever required, for example during charging of the device (within adocking station for example). The linear calibration may advantageouslybe performed quickly, each image acquisition, or at device start-up tocompensate for minor deviations.

The local calibration is first described as a first embodiment andsecond embodiment. The broad calibration is described as a thirdembodiment of the present invention. Note that the broad calibration maynot be required for every type of device. Some devices according to thepresent invention show a linear range over the entire range of themoving part, and thus such a calibration may not be required.

In the first embodiment according to the present invention, in order tocalibrate the focusing means by controlling the position of the focus ata given wavelength, a calibration pattern may be positioned at a knownposition along the optical axis of the image acquisition device, e.g.sensor and within its field of view. The pattern can for example beplaced at the edge of the FOV so as not to disturb the images. Theimages presented to the users may even not comprise the image of thecalibration pattern so as not to disturb the user, but only the centralpart, comprising the object of interest.

As the focus position or focal length can depend on the wavelength, eachlight source, having a different spectral bandwidth centered on adifferent wavelength will in this case have different focus positionsfor a same configuration of the device (same curvature of the liquidlens or same position of the image acquisition device), or a differentconfiguration for a same focus position. Thus, the followingcalibrations are preferably performed for each wavelength. However, thecalibration can also be performed at one wavelength, for instance theshortest wavelength, and then the calibration extrapolated to all theother wavelengths of the device, by means of a formula, a look up table,an algorithm, etc.

As the calibration can be wavelength dependent, and the light sourceshave a narrow spectral bandwidth, the chromatic aberrations associatedto each light source can be negligible. As the calibration measurementsare performed separately for each wavelength, chromatic aberrations arecorrected for each wavelength.

An inspection device according to the present invention can be used toanalyze an object of interest at the skin surface and below the skinsurface. Calibration, is used to be able to know at which depth themeasurement is being performed. As there can be medically relevantinformation in an entire volume of skin, the inspection device accordingto embodiments of the present invention can map structures at differentdepths accurately. This first embodiment aims at calibrating the deviceat a known fixed distance from the image acquisition device, above or atthe level of the skin surface with a known calibration pattern, thusrecord the driving level for which the device is focused in said knownfixed position at one of the wavelengths.

For example, a calibration pattern can be placed directly onto one orboth of the upper and/or lower surface of the front glass 14. Suchcalibration pattern is at a fixed position with respect to a referenceviewing surface of the device. In this case the viewing surface is thelower surface of the front glass 14. This can be used to calibrate theimage acquisition device, e.g. camera in absolute measures and provide areference point to determine the focus depth. By putting the calibrationpattern onto the glass, no additional parts have to be added and/orcorrectly positioned, when performing the calibration. This embodimentprovides a fixed distance between the calibration pattern and the imageacquisition device, e.g. camera sensor (or between the calibrationpattern and the magnifying lens, in case of using a moving imageacquisition device, e.g. sensor).

The advantage of providing a calibration pattern on the lower surface asreference viewing surface which is to be placed in contact with the skinin use is that the device is then calibrated for the skin surface.Additionally, the calibration pattern can also be provided on the insidewall of the housing and in the field of view of the image acquisitiondevice. It is also possible to mark with a stamp seal, or provide atemporary tattoo on the skin surface of a patient and to use the areawith the stamp or tattoo for calibrating the device before performing anacquisition.

In one embodiment of the present invention, a calibration pattern can beprovided directly onto the front glass. FIG. 3 a ) shows examples ofcalibration patterns. FIG. 3 b ) is the same as FIG. 2 with the additionof a pattern 52 on the lower surface of the front glass. The pattern 52has been enlarged for clarity: these types of patterns are mostlymanufactured by thin film deposition and the individual lines can haveorders of magnitude down to the sub-micrometre.

The pattern 50 shown of FIG. 3 a ) has the advantage that it does notonly provide information on the focus depth, but also information on theresolution by computing the modulation transfer function of the opticalsystem with the pattern. However, the invention is not limited to suchpatterns. More simple patterns may also be used, such as simple lines orcircles.

The calibration pattern can also be integrated within the front glass atan arbitrary depth from the front glass surface. If the distance to theimage acquisition device, e.g. camera sensor (or lens, in case of usinga moving image acquisition device, e.g. sensor) is known, the distancefrom the front glass surface can be compensated for. The pattern can inprinciple be positioned anywhere within the field of view of the camera,for example on the housing material itself provided its position givesis reproducible and provides reproducible focus depths when used forcalibration.

The calibration process can be implemented by letting a camera autofocusarrangement obtain a sharp image of the pattern on the front glass (forexample by implementing an edge detection method) at a given wavelength.Since the distance between the pattern and the image acquisition device,e.g. camera sensor is known, the corresponding digital driving level canbe stored as a reference driving level of an absolute focus depth forthe given wavelength.

The positions and distance from the calibration pattern to the frontglass surface used as a reference viewing surface is important. Thecalibration procedure records the digital driving levels when thecalibration pattern is sharp. This value can be extrapolated to find therelationship of digital driving levels to focus on depths deeper intothe skin. This can be done from known characteristics of the device orby finding the calibration data with an external object having therequired depth.

For image acquisition devices, e.g. cameras which are provided with amoving sensor together with a static lens, the distance between the lensand the sensor can be used. If the calibration pattern is placed on thelower surface of the front glass, and an object is placed in contactwith the front glass as the viewing surface, the part of the objectsurface touching the front glass will be in focus when the referencedriving level is used. If the device is calibrated for a given distanceand a given wavelength, the reference driving level is known. The imageacquisition procedure may start at all spectral bandwidths as the focallength can be extrapolated from one wavelength to all other wavelengths(or spectral bandwidths). Thus, images of the object at a plurality ofdepths may be acquired.

In a second embodiment of the present invention, a second pattern isprovided on or in the front glass as the reference viewing surface at adistance from the first pattern. For example, this can be implemented byproviding the patterns on the upper and lower surfaces of the frontglass, in which case they are separated by the thickness of the glass.FIG. 4 shows a) a top view and b) a side view of two patterns arrangedat opposite sides of the front glass. A pattern 51 is provided on theupper surface of a front glass 52 on the front side 53. Another pattern54 is put on the back side 55 of the front glass 52. By knowing thethickness of the glass it is possible to establish a relationshipbetween the positions of the patterns 51 and 54 and the digital drivinglevels, assuming linearity of the system or aknown/pre-defined/approximated non-linear behavior of the system. Withthis knowledge it is possible to steer the image acquisition device,e.g. camera to focus on a specific depth in the sample, and each imagecan be related to the absolute depth in the object where it wascaptured.

In principle, the patterns can be provided at an arbitrary distance toeach other in the substrate. The distance to and within the skin isimportant. The distance to the static image acquisition device, e.g.camera or the static lens is an offset to this distance.

The patterns absolute positions in the substrate should at reproduciblepositions. The offset distances to the static image acquisition device,e.g. camera sensor or the static lens should be constant. It ispreferred that the distance between them is larger than the depth offocus of the camera, in order for the image acquisition device, e.g.camera to differentiate between them. On the other hand thicker glasswill result in geometric distortions. Thus, a preferred range ofthickness for the front glass or substrate is about 0.8-1.4 mm, forexample 1.1 mm, for a depth of focus of approximately 0.6 mm.

In the example shown on FIG. 11 , the width of the tip of the device isin the range of 15 to 30 mm which can be too wide to access certainparts of the body for inspection, such as between fingers or toes. Itcan be beneficial to add a cone shaped extension piece with a narrowertip to the dermatoscope, to better match the shape of the body to beexamined.

FIG. 5 shows an embodiment of the present invention comprising extensionpieces that can be put in front of the front glass of the dermatoscope.FIG. 5 a ) comprises the housing 20 an image acquisition device, e.g.camera having a camera lens 21, a camera sensor 26, a short cone 22 anda front glass 24. The cone 22 may be permanently integrated to thehousing 20. If the length 25 of the extension piece 22 is moderate, thecamera lens can be adapted to compensate for the additional length 25.However, for long extension pieces this might no longer be possible andan additional lens may be required. FIG. 5 b ) shows an example of a“longer” type of extension cone. It comprises the housing 30, an imageacquisition device, e.g. a camera with a camera lens 31, a camera sensor36, a long cone 32, a cone lens 33 and a front glass 34. The long cone32 can be shaped so that it fits in more narrow angles of the body, suchas for example between fingers or toes.

In embodiments where the regular cone can be removable and replaceableby an extension cone, then there would always be only one glass in frontof the lens (as depicted in the current pictures). However, if theregular cone is not removable, then there will be an additional glassbetween the regular cone and the extension cone. The extension cone isbasically added on top of the regular cone. This has the additionalbenefit of preventing dust or similar from getting into the device . . ..

If a non-linear type of focusing means is used, such as a camera lens,for example a liquid lens, this shift might force the focus to operatein a part of its range that is more difficult to manage, e.g. where asmall change in voltage might yield a big change in focal length orwhere an increase in voltage reduces the focal length. The main goal ofthe present application is to operate in the part of the range for whichthe step size is smallest. FIG. 6 shows the relation between the focallength (on the y axis) and the applied voltage (on the x axis) for aliquid lens. For example, at focal lengths around 0.5-0.7 mm, smallvoltage steps will result in small changes in the focal length, whichcan make fine tuning easier. While for example at 1 mm, a small changein voltage yields a large step in focal length. However, in the lowerpart of the range 0.0 to 0.4 V, an increase in voltage results in adecrease in focal length. For some applications, it is desirable to keepthe liquid lens operating at lower focal lengths, in an approximatelylinear part of the range. If an extension piece is used, it can bebeneficial to insert a lens inside the piece, for example lens 33 inFIG. 3 b ), to reduce the focal length. The lens can be a conventionallens of simpler type than a liquid lens.

If an extension piece is used, the calibration pattern should be put atthe front glass of the extension piece as the reference viewing surface.

In a third embodiment according to the present invention there isforeseen an additional three-dimensional calibration piece which is putin front of the front glass as reference viewing surface. Thethree-dimensional calibration piece can be any substrate, such as glass,plastic, etc. It can be transparent but this is not a requirement as itis removed when real measurements are performed. The pattern comprises aphysical structure that is provided at a variety of known depths withinthe calibration piece (that can exceed the thickness of the frontglass). A simple solution can for example be a substrate with straightparallel lines, which are in a plane which is tilted with respect to theplane of the image acquisition device, e.g. camera sensor. FIG. 7illustrates an exemplary embodiment comprising a pattern 61 havingparallel lines extending at different depths (z direction) of thesubstrate, the z direction corresponding to the optical axis of thedevice. The sample can be implemented by e.g. engraving such as laserengraving on a substrate, micro-lithography of a metal sheet, glassetching, milling, printing with ink, 3-dimensional printing orsub-surface laser engraving, but not limited thereto. The test patterncan be retrofitted and applied as a sticker on the cover glass ofexisting devices. The outer boundaries 60 can e.g. be the outerboundaries of a substrate. The substrate can be a stand-alone element,for example a metal sheet, where no surrounding material is necessary.

Assuming FIG. 7 comprises a substrate 60, a part 62 of the calibrationpattern can extend horizontally (along the x-axis) in the surface 68,while other parts may be completely imbedded inside the substrate 60,for example the inclined horizontal lines 63 or the “flat” horizontallines 64. Thus, the lines of the calibration pattern have a V shape witha flat and horizontal section in the vertex 62 and at the extremities64. There can also be lines 65 in the vertical direction (along the yaxis) which do not cross any of the horizontal lines 62, 63 or 64 in theflat sections. The pattern can be symmetric around the vertex 62.

The substrate or substrate with the three-dimensional pattern, e.g.engraved pattern can be attached or easily snapped or fixed bymechanical means for example to the front glass of the dermatoscope oran extension cone so that the upper surface 68 (or 62) of the substrate60 is in contact with the lower surface of the front glass of thedermatoscope (or of the extension cone, in either case being thereference viewing surface) and the side 67 is facing outwards.

A reference level is considered to be at minimum level, e.g. 0 mm ofdepth. In FIG. 7 this reference plane coincides with the upper surface68 of the substrate 60. The maximum level of depth of the pattern orstructure 61 is indicated with arrow 65. The distance between thecalibration lines of the pattern can be in the sub-mm range, for example20-40 um. The lines can extend continuously across the substrate in thex-direction. The continuous lines make it possible to have a calibrationpattern available for every driving level position of the camera focusfor a given wavelength. First the camera can be made to focus with agiven wavelength on a certain level, for example along the dashed line69. The driving level is kept in memory and a relative focus function iscalculated (based on for example edge detection) to give the horizontalposition of the focus. In FIG. 7 the horizontal lines 63 can bedescribed by a linear function z=kx+x₀ (initial), and by knowing thedistance 66, the x₀ (initial) can be determined. And using the x valuefrom the relative focus function, the depth can be calculated for thedriving level in question. This procedure can be repeated for a multipleof different x values like 69. Note that due to the V shape of thethree-dimensional calibration pattern, the measurements can be performedtwice, and the average could be used as a result.

The pattern like the one in FIG. 7 may also further provide informationon the resolution and MTF (Modulation Transfer Function) and theresolution of the camera system.

The substrate may also comprise numbers engraved inside wherein eachnumber indicates the actual depth inside the substrate.

The calibration pattern can also include coloured elements so that thecalibration pattern can be used for colour calibration. The calibrationpattern may be a colour chart or a colour patch.

In another embodiment, instead of using a substrate, a phantom of humanskin can be used to calibrate the sensor, e.g. a pig skin or anartificial piece of skin manufactured to be a phantom. Advantages arethat such a phantom is more realistic and may improve the calibration.Such a phantom may only be used in the factory. As pig skin is verysimilar to human skin, a sample of pig skin may also be used as acalibration substrate for the device of the present invention. A 3Dartificial skin tissue can be made as described in US 2016/0122723 whichis incorporated herein by reference.

FIG. 8 shows a flow chart describing an embodiment of the presentinvention where the image acquisition device, e.g. camera is calibratedusing the structure in FIG. 7 . In step 80, the image acquisitiondevice, e.g. camera setting is determined when it is focused on thevertical lines 65 at 0 mm of depth, the reference level. In step 81, theimage acquisition device, e.g. camera is being focused on the verticallines 65 at a maximum depth, and the image acquisition device, e.g.camera settings are recorded. In step 82 the focus depth is set to anarbitrary depth between the minimum depth (80) and the maximum depth(81), in step 83, the current focus setting the relative focus functionfor the group of inclined lines 63 is determined. And in step 84, afocus function F(x) is calculated relative to the horizontal lines:

${F(x)} = \frac{\sum_{y}\left\lbrack {{f(y)} \otimes {i\left( {x,y} \right)}} \right\rbrack^{2}}{\sum_{y}\left\lbrack f_{center} \right\rbrack^{2}}$

Where i(x, y) is the normalized image pixel value at location (x, y),f(y) is a one-dimensional high-pass filter kernel with center tap valuef_(center) and sum of the coefficients 0 (for example (−0.5, 1, −0.5))and ⊗ is the convolution operator. In step 84, “Determine the positionalong the x-axis where the relative focus function in 83 is at itsmaximum value” and finally in step 85 performs the step: “Insert x fromstep 84 and calculate the depth z”.

FIG. 9 shows an example of a relative focus function 91 overlaid onto acalibration pattern 90 (see Boddeke et. al., Journal of Microscopy vol.186, Pt3 June 1997, pp 270-274).

The graph has the x-position on the x-axis and the relative focus valueon the y-axis. The position of sharpest focus is considered to be at thetop of the curve 92. This also corresponds to the visual appearance ofthe calibration pattern. The view of pattern 90 corresponds to lookingstraight onto the surface 68 in FIG. 8 .

The present invention also provides the possibility of providing atemperature sensor in proximity of the liquid lens. As the drivingvoltage of the liquid lens, or the calibration of the liquid lens isdependent on temperature variations, such a temperature sensor can beused to compensate for variations in the focus as a function oftemperature.

In one embodiment of the present invention, an additional pattern canused externally from the dermatoscope, for example in a charging stationor docking station. A pattern can be for example lines for focuscalibration (as described previously), but this could also be forexample color patches with known color for color calibration.

FIG. 10 shows a schematic illustration of a part of the dermatoscope100, having a front glass 101 (as reference viewing surface) providedwith a calibration pattern 102. The part 100 of the dermatoscope can beinserted into the charging station 103 so that a second calibrationpattern 104 can be detected via the front glass 101 of the dermatoscope.The position of the part 100 of the dermatoscope when inserted insidethe charging station 103 can be known so that the distance between thecalibration patterns 102 and 104 can be used for calibration. Inprinciple, the second pattern 104 can be positioned anywhere within thefield of view of the camera of the dermatoscope.

One aspect of the present invention is to provide a dermatologicalinspection device. Such a device can be used to diagnose skin cancerssuch as melanomas.

Traditionally five signs have used by dermatologists to classifymelanomas, “ABCDE” for

-   -   Asymmetry,    -   irregular Borders,    -   more than one or uneven distribution of Color,    -   a large Diameter (greater than 6 mm) and    -   the Evolution of the moles.

In addition for nodular melanomas a different classification, EFG, isused which are

-   -   Elevated: the lesion is raised above the surrounding skin.    -   Firm: the nodule is solid to the touch.    -   Growing: the nodule is increasing in size.

Thus, the elevation above the skin may also be used as means to detectand/or classify skin cancer lesions. It may also happen that such skinlesions are sunken under the skin level, as a cavity, for example in thecase of melanoma ulceration.

In order to analyze the three-dimensional shape of skin lesions whichmay be cancer, a volume reconstruction algorithm based on shadows andreflection may be used.

A review of existing methods on Shape reconstruction from Shadows andReflections: “Shape Reconstruction from Shadows and Reflections”, Thesisby Silvio Savarese, 2005, California Institute of Technology. Thefollowing article by the same author also provides information on ShapeReconstruction, published in International Journal of Computer Vision,March 2007, Volume 71, Issue 3, pp 305-336, “3D Reconstruction by ShadowCarving: Theory and Practical Evaluation”, by Silvio Savarese et. Al.

Different algorithms exist in order to reconstruct the shape of anobject from its shadow. It is an object of the present invention toincorporate such a method into a dermatological inspection device. Norestrictions are anticipated for the skilled person to find a suitableprocedure to provide a 3D volume reconstruction based on shadows.

One of the first known algorithms proposed is called Shadow Carving. Thealgorithm uses the information of a cone by a point of observation andthe silhouette in an image obtained from that point.

By using different viewpoints and intersecting the cones from thesevarious viewpoints, the estimate of the object can be reconstructed.However, this method is not capable or reconstructing concavities in anobject, which may be the case in the present invention.

Shafer and Kanade (“Using shadows in finding surface orientations”,Computer Vision, Graphics, and Image Processing 22:145-176. 1983)established fundamental constraints that can be placed on theorientation of surfaces, based on the observation of the shadows onesurface casts onto another. Embodiments of the present invention can usereconstruction methods where the light source positions are known. Also,reconstruction methods used image self-shadows, i.e. shadows cast by theobject e.g. a skin lesion such as a melanoma upon itself and not shadowscast by other objects. A further assumption that can be made is thatcontour of the object, e.g. a skin lesion such as a melanoma, is definedby a smooth function and that the beginning and end of each shadowregion can be found reliably. Each pair of points bounding a shadowregion yields an estimate of the contour slope at the start of theshadow region, and the difference in height between the two points. Forexample, the information from shadows from images taken with a series oflight sources at different positions can be used to obtain aninterpolating spline that is consistent with all the observed datapoints.

Hence, in reconstructing a three-dimensional object from its shadowsaccording to this embodiment of the present invention is to knowprecisely the position of the image acquisition device, e.g. imagingsensor, which is the viewpoint, and to know precisely the position ofthe light sources illuminating the object.

In order to be able to reconstruct the shape of a skin lesion with thedevice of the present invention, a plurality of light sources are addedto the inspection device in addition to the first ring of light sources1110. These additional light sources 1100 are arranged in a second ringso as to be in a proximity of the region of interest at the skin surfaceand so as to illuminate the region of interest. These light sources 1100used for shadowing thus provide a horizontally directed illumination.These light sources are to be combined with the existing light sources1110 which provide a more vertical illumination. Hence multiple lightsources provide illumination of a surface skin lesion from perpendiculardirections.

The inspection device according to these embodiments of the presentinvention further comprises a second ring of light sources 1100 providednear the front plate of the device according to the present invention inaddition to the first ring of lights sources 1110. These light sources1100 of the second ring provide a substantially horizontal illuminationdirected towards the region of interest at the surface of the skin, whensaid region of interest is in the middle of the ring of the tip of thedevice.

In a preferred embodiment according to the present invention, the ring1100 of light sources provided in a proximity of the front platecomprises at least 4 light sources, more preferably 8 light sources andmore preferably 16. As the tip of the inspection device is rectangularin an embodiment of the present invention, the light sources 1100 can bearranged in a rectangular ring in multiples of four. Knowing the exactposition of each light source 1100, 1110 is advantageous to be able toreconstruct the three-dimensional shape of the object of interest.

As these light sources 1100 provide mostly a horizontal illumination,they are mainly used to cast shadows generated by protrusions extendingabove the skin surface. In an embodiment, these light source 1100 can bewhite LEDs.

Four light sources arranged in the second ring at 90° one from anothersurrounds the region of interest, however there is no redundantinformation. Ideally more light sources are provided to increase thenumber of shadows. A total of 8 or 16 light sources 1100 in the secondring in combination with seven or fourteen light sources 1110 in thefirst ring is a good compromise to provide enough information to be ableto reconstruct the shape of the skin lesion. However, more light sourcescan also be used. Increasing the number of light sources increases theprecision of the measurements but increases the processing time. Thus,in a preferred embodiment of the present invention, 12 to 25 lightsources can be be used in the first and second rings providinghorizontal and vertical illumination.

The image capturing device or sensor is located at approximately 70 mmfrom the front plate in the dermatoscope shown in FIG. 11 .

If the object of interest such as a skin lesion comprises cavities, orconcave regions, the light sources 1100 are capable of generating ashadow which corresponds to such a cavity but the cavity cannot beviewed. Therefore, the light sources 1110 of the first ring, which in anembodiment are color LEDs can be used. In the device according to anembodiment of the present invention, the first LED ring is located at 35mm from the front plate. These will enable illuminating the skin lesionvertically and also the viewing of cavities in the region of interestand to assist in the reconstruction. In the example shown on FIG. 11 ,there are two arrays of seven LEDs, thus seven plus 16 or 23 total.

To generate sufficient images of shadows, the light sources 1110 and1100 of the first and second rings are illuminated sequentially togenerate images with the required shadows. The image acquiring devicethen acquires an image for each sequential illumination. For example,first the light sources of the second ring which are provided inproximity to the front surface are lit sequentially and then the lightsources 1110 are lit sequentially providing illumination parallel to theoptical axis.

Two light sources whose spectral bandwidth do not overlap or have anegligible overlap may acquire images simultaneously in order to reducethe acquisition time.

Given the large amount of images generated, which corresponds to thenumber of light sources, it is preferable to transfer the imagesacquired to a processor which has processing means to analyse theshadows of each image and reconstruct the three-dimensional shape of theobject of interest.

To be able to penetrate the skin, high power LEDs are preferred. Inorder to achieve good image quality, it is necessary that each of theindividual narrowband illuminations can be set to an output power whichis sufficient (but also not too much) for the image acquisition device,e.g. sensor. With too low power, the noise will be very high for thatspectral image, with too much power then there will be clipping andimage quality will be bad as well. The present invention makes itpossible to select the correct light output for each LED individually tomatch with the image acquisition device, e.g. sensor.

It is thus advantageous to be able to control the relative power of theindividual narrow band illuminations independently. The setting of therelative power of the individual light sources depends on many factorssuch as the skin type of the patient, the specific lesion being imaged(red lesion, white lesion, . . . ) as well as the spectralcharacteristics of the image acquisition device, e.g. sensor.

Embodiments of the present invention provide means to modulate the powerof the individual LED light sources relatively to each other, such thatfor a given (selected) gain/offset setting of the sensor, a good imagequality can be achieved for all spectral images simultaneously. Forexample, it is possible to optimize the exposure, which can be thecombination of exposure time (e.g. shutter speed), lens aperture andlight intensity, for each light type. In doing so, a higher/optimal SNRand contrast can be obtained.

Also for polarized images the power of the polarized LEDs preferably isdoubled (modulated) such that they can be combined with acquisitions ofunpolarized light sources.

The aim of the device according to embodiments of the present inventionis to acquire at each wavelength (thus seven times if there are seventypes of light sources), a plurality of images, for example 5 to 10,each with a different depth into the skin in order to obtain informationabout the skin lesion at a plurality of depths, at each wavelength used.However, to acquire such a large number of images, thus 35 to 70 images,requires a lot of time. The number of frames per second is approximately30. Thus, the acquisition of all the image data requires 1 to 2 seconds.A problem associated with long acquisition times is that a patient movesand thus the images suffer from motion artifact.

The imaging time can be decreased by acquiring several images, at thesame depth of focus, with different spectral bands in parallel. Forexample: acquiring a red and blue image at the same time. This howeverrequires to take into account the emission spectrum of the LEDs, and thefilter spectrum of the image acquisition device, e.g. sensor to avoidcrosstalk between the images taken simultaneously with different lightsources. Hence, the images can be captured as long as there is nocrosstalk (or predictable crosstalk which can be calculated).

The image acquisition device, e.g. light sensor for these embodiments ofthe present invention can comprise a plurality of channels (see FIG. 15), each channel being configured to capture light in a differentoverlapping spectral bandwidth. As an example, consider a multi-channelsensor having the spectral sensitivity shown on FIG. 12 and r(λ)represents the spectral sensitivity for the red channel or sub-pixel,g(λ) represents the spectral sensitivity for the green channel orsub-pixel and b(λ) the spectral sensitivity for the blue channel orsub-pixel.

The full spectral bandwidth of each light source is used when images areacquired. The small amount of noise/cross talk is removed bycompensation. For example: The contribution of green sensor from theblue LED. The spectral power distribution of the light sources accordingto an embodiment of the present invention is shown on FIG. 13 .

It is advantageous to take into account the type of skin which is beinglit by the inspection device as the spectral bandwidth of the reflectedobject is different. For example, the skin can be skin with 7.5%melanomas in the epidermis, which corresponds to the middle curve ofFIG. 14 . The reflectance spectrum of the skin can be represented with areflectance spectrum R_(object) (λ), and is a function of thewavelength.

In order to explain this embodiment of the present invention, thecontribution of each light source within each channel of the imageacquisition device, e.g. sensor is calculated for a reference objecthaving a reflectance spectrum R_(object)(λ).

In a first step, one can calculate the output of the red, green and bluechannels of the sensor when the reference object is illuminated with afirst light source having a first spectral bandwidth BW1, for example ablue light source having a relative spectral power distributionS_(B)(λ):

R_(Blue) = ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)(λ) * r(λ)G_(Blue) = ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)(λ) * g(λ)B_(Blue) = ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)(λ) * b(λ)

R_(blue) represents output in the red channel of the sensor, and iscalculated as the convolution between the spectral bandwidth associatedto the red channel and the relative spectral power distribution S_(B)(λ)of the light source convolved with a reflectance spectrum R_(object)(λ)of a reference object. It represents the overlap between the relativepower spectrum of the reflected illumination on the reference object andthe spectral bandwidth of the red channel of the sensor.

Similarly, G_(Blue) and B_(Blue) represent respectively the output ofthe green and blue channel of the sensor when the reference object isilluminated with a blue light source.

Thus, when the power spectrum of the light source and the spectralbandwidth of the channel have the widest overlap, the output in thatchannel will be the greatest.

In this example, the output of the blue channel of the light sensor willbe the greatest when the object of reference is illuminated with a bluelight source.

Introducing the following ratios, which represent the ratio of bluelight (from the blue light source) captured by each sub-pixel, or eachchannel, respectively the red, the green and the blue:

${r_{Blue} = \frac{R_{Blue}}{R_{Blue} + G_{Blue} + B_{Blue}}},{g_{Blue} = \frac{G_{Blue}}{R_{Blue} + G_{Blue} + B_{Blue}}},{b_{Blue} = \frac{B_{Blue}}{R_{Blue} + G_{Blue} + B_{Blue}}}$

The ratio of blue light captured by the blue sub-pixels b_(Blue) is thelargest, for example 90%, as the overlap of the spectral bandwidthbetween the spectral sensitivity of the blue sub-pixel and the blue LEDlight spectrum is the largest. The ratio of blue light captured by thegreen sub-pixel a g_(Blue) is thus the second largest, for example 8%and the ratio of blue light captured by the red sub-pixel, r_(Blue), isthus the smallest for example only 2%.

Similarly, the output of each channel when a second light source havinga second spectral bandwidth BW2, for example a red light source havingrelative spectral power distribution S_(R)(λ) illuminates the samereference object, is provided by:

R_(Red) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)(λ) * r(λ)G_(Red) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)(λ) * g(λ)B_(Red) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)(λ) * b(λ)

In this example, G_(Red) represents the output of the green channel ofthe sensor when the object is illuminated with the red light source.Similarly, B_(Red) represents the output of the blue channel when theobject of reference is illuminated with a red light source.

Again, one can calculate the ratios expressing the amount of red lightcaptured by each sub-pixel, or each channel:

${r_{Red} = \frac{R_{Red}}{R_{Red} + G_{Red} + B_{Red}}},{g_{Red} = \frac{G_{Red}}{R_{Red} + G_{Red} + B_{Red}}},{b_{Red} = \frac{B_{Red}}{R_{Red} + G_{Red} + B_{Red}}}$

In this case, the ratio of red light r_(Red) absorbed by red sub-pixelsis assumed to be the largest, for example 92%, g_(Red) the secondlargest, for example 7% and b_(Red) the smallest ratio, for example only1%.

It is now assumed that a different, but similar object (i.e. similarskin type), having reflectance spectrum R′_(0bject)(λ), issimultaneously illuminated with the first and second light sourceshaving first and second spectral bandwidth BW1 and BW2, in the example ared and blue light source, then the obtained red, green and blue sensoroutput are given by, assuming the system is linear:

R_(Red + Blue)^(′) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)^(′)(λ) * r(λ) + ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)^(′)(λ) * r(λ)G_(Red + Blue)^(′) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)^(′)(λ) * g(λ) + ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)^(′)(λ) * g(λ)B_(Red + Blue)^(′) = ∫₄₀₀⁷⁰⁰S_(R)(λ) * R_(object)^(′)(λ) * b(λ) + ∫₄₀₀⁷⁰⁰S_(B)(λ) * R_(object)^(′)(λ) * b(λ)

This can be rewritten as:R′ _(Red+Blue) =r′ _(Red+Blue) *(R′ _(Red+Blue) +G′ _(Red+Blue) +B′_(Red+Blue))G′ _(Red+Blue) =g′ _(Red+Blue) *(R′ _(Red+Blue) +G′ _(Red+Blue) +B′_(Red+Blue))B′ _(Red+Blue) =b′ _(Red+Blue) *(R′ _(Red+Blue) +G′ _(Red+Blue) +B′_(Red+Blue))

Each term can also be developed as:

$\begin{matrix}{R_{{Red} + {Blue}}^{\prime} = {\frac{r_{Red} + r_{Blue}}{r_{Red} + r_{Blue}}*r_{{Red} + {Blue}}^{\prime}*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}} \\{= {\left( {{\frac{r_{Red}}{r_{Red} + r_{Blue}}*r_{{Red} + {Blue}}^{\prime}} + {\frac{r_{Blue}}{r_{Red} + r_{Blue}}*r_{{Red} + {Blue}}^{\prime}}} \right)*}} \\{\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}\end{matrix}$$G_{{Red} + {Blue}}^{\prime} = {\left( {{\frac{g_{Red}}{g_{Red} + g_{Blue}}*g_{{Red} + {Blue}}^{\prime}} + {\frac{g_{Blue}}{g_{Red} + g_{Blue}}*g_{{Red} + {Blue}}^{\prime}}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$B_{{Red} + {Blue}}^{\prime} = {\left( {{\frac{b_{Red}}{b_{Red} + b_{Blue}}*b_{{Red} + {Blue}}^{\prime}} + {\frac{b_{Blue}}{b_{Red} + b_{Blue}}*b_{{Red} + {Blue}}^{\prime}}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$

The individual contribution of each light source type to each channelcan now be estimated with the following relations, using the equationsdeveloped above:

$R_{Red}^{\prime} = {\left( {\frac{r_{Red}}{r_{Red} + r_{Blue}}*r_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$R_{Blue}^{\prime} = {\left( {\frac{r_{Blue}}{r_{Red} + r_{Blue}}*r_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$G_{Red}^{\prime} = {\left( {\frac{g_{Red}}{g_{Red} + g_{Blue}}*g_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$G_{Blue}^{\prime} = {\left( {\frac{g_{Blue}}{g_{Red} + g_{Blue}}*g_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$B_{Red}^{\prime} = {\left( {\frac{b_{Red}}{b_{Red} + b_{Blue}}*b_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$$B_{Blue}^{\prime} = {\left( {\frac{b_{Blue}}{b_{Red} + b_{Blue}}*b_{{Red} + {Blue}}^{\prime}} \right)*\left( {R_{{Red} + {Blue}}^{\prime} + G_{{Red} + {Blue}}^{\prime} + B_{{Red} + {Blue}}^{\prime}} \right)}$wherein each individual term is known from a calibration performed withthe reference object having reflectance spectrum R_(object) (λ) and frommeasurements performed with the simultaneous illumination with the redand blue light sources.

Note that the accuracy of this technique is mainly determined by thesimilarity of the reflectance spectrum of the reference object and themeasured object, i.e. of R_(object) (λ) and R′_(object) (λ) are. In theabove example the object to be lit by multi-spectral light sources isassumed to be a certain skin type. If the actual skin type is verysimilar, then the obtained accuracy will be quite high.

If the object to be lit by the multi-spectral light sources is notknown, then this technique could still be used by assuming a certainreflectance spectrum of the object, for example a flat spectrum.

As can be appreciated from the above the present invention provides amethod for retrieving a first and a second spectral image of amulti-spectral image of an object illuminated simultaneously with afirst and second light source of a dermatological inspection device, thespectral bandwidth of the light sources being comprised in the spectralsensitivity range of a light sensor. The method comprises the steps of

-   -   Illuminating an object having a known reflectance spectrum        R′_(object)(λ) with the first and the second light source having        a first BW1 and a second BW2 substantially distinct spectral        bandwidths,    -   Acquiring an image with the light sensor having a plurality of        channels, each channel configured to capture light in a        different overlapping spectral bandwidth,    -   Retrieving each channel of the first and second spectral image        from each channel of the acquired multi-spectral image and from        a pre-calibrated ratio expressing the convolution between the        spectral bandwidth associated to the channel and the first or        second spectral bandwidth convolved with a reflectance spectrum        R_(object)(λ) of a reference object.

Each channel C′_(j) _(BW1) and C′_(j) _(BW2) of the first and secondspectral image is obtained by calculating each channel j of the spectralimage for the first and second spectral bandwidth

$C_{j_{{BW}\; 1}}^{\prime} = {\left( {\frac{c_{j_{{BW}\; 1}}}{c_{j_{{BW}\; 1}} + c_{j_{{BW}\; 2}}}*c_{j_{{{BW}\; 1} + {{BW}\; 2}}}^{\prime}} \right)*\left( {\sum\limits_{i = 1}^{n}C_{i_{{{BW}\; 1} + {{BW}\; 2}}}^{\prime}} \right)}$$C_{j_{{BW}\; 2}}^{\prime} = {\left( {\frac{c_{j_{{BW}\; 2}}}{c_{j_{{BW}\; 1}} + c_{j_{{BW}\; 2}}}*c_{j_{{{BW}\; 1} + {{BW}\; 2}}}^{\prime}} \right)*\left( {\sum\limits_{i = 1}^{n}C_{i_{{{BW}\; 1} + {{BW}\; 2}}}^{\prime}} \right)}$

-   -   wherein n is the number of channels of the light sensor, and        C_(j) _(BW1) is expressed by

${c_{j_{{BW}\; 1}} = \frac{C_{{jBW}\; 1}}{\sum\limits_{i = 1}^{n}C_{i_{{BW}\; 1}}}},{c_{j_{{BW}\; 2}} = \frac{C_{{jBW}\; 2}}{\sum\limits_{i = 1}^{n}C_{i_{{BW}\; 2}}}}$

-   -   and wherein the outputs C_(j) _(BW1) and C_(j) _(BW2) of channel        j when the reference object of reflectance spectrum        R_(object)(λ) is illuminated by a light source having spectral        bandwidth BW1 and BW2, is expressed by        C _(j BW1)=∫_(s1) ^(s2) S _(j)(λ)*R _(object)(λ)*bw1(λ) and C        _(j BW2)=∫_(S1) ^(s2) S _(j)(λ)*R _(object)(λ)* bw2(λ)    -   wherein S_(j)(λ) is the spectral sensitivity of channel j and        bw1(λ) and bw2(λ) are the relative power spectrum of the first        light source and second light source respectively, and s1 and s2        correspond to the lower and upper limits of the spectral        sensitivity of the sensor.

The sensor can be an RGB sensor comprising three channels, e.g.respectively a red, a green and a blue channel.

In an embodiment of the present invention any of the methods describedabove suitable for use by an inspection unit can be implemented by adigital device with processing capability including one or moremicroprocessors, processors, microcontrollers, or central processingunits (CPU) and/or a Graphics Processing Units (GPU) adapted to carryout the respective functions programmed with software, i.e. one or morecomputer programs. The software can be compiled to run on any of themicroprocessors, processors, microcontrollers, or central processingunits (CPU) and/or a Graphics Processing Units (GPU).

Such a device may be a standalone device such as a docking station ormay be embedded in another electronic component, e.g. on a PCB board.The device may have memory (such as non-transitory computer readablemedium, RAM and/or ROM), an operating system, optionally a display suchas a fixed format display such as an OLED display, data entry devicessuch as a keyboard, a pointer device such as a “mouse”, serial orparallel ports to communicate with other devices, network cards andconnections to connect to a network.

The software can be embodied in a computer program product adapted tocarry out the following functions when the software is loaded onto therespective device or devices or any other device such as a networkdevice of which a server is one example and executed on one or moreprocessing engines such as microprocessors, ASIC's, FPGA's etc.

The software can be embodied in a computer program product adapted tocarry out the following functions, when the software is loaded onto therespective device or devices, and executed on one or more processingengines such as microprocessors, ASIC's, FPGA's, etc:

-   -   establishing a relationship between first adjustments of        focusing means of the unit and the positions of the focus points        along the optical axis of the unit, and    -   storing a second adjustment wherein the focus position is at a        fixed known position of a calibration pattern,

the step of storing a second adjustment comprising the step of analysingthe calibration pattern at the known fixed position with an edgedetection algorithm.

The software can be embodied in a computer program product adapted tocarry out the following functions, when the software is loaded onto therespective device or devices, and executed on one or more processingengines such as microprocessors, ASIC's, FPGA's, etc:

-   -   Using a three-dimensional continuous calibration structure        having a known extension in space which is placed at a known        position with respect to the reference position,    -   instructing the image capturing device to establish an absolute        reference value, engage a certain driving level relative the        absolute reference value and use a focus function to obtain the        coordinates corresponding to said driving level, using the        calibration structure's extension in space to calculate the        depth for the chosen driving level.

Any of the software mentioned above may be stored on a non-transitorysignal storage means such as an optical disk (CD-ROM, DVD-ROM), magnetictape, solid state memory such as a flash drive, magnetic disk such as acomputer hard drive or similar.

While the invention has been described hereinabove with reference tospecific embodiments, this was done to clarify and not to limit theinvention. The skilled person will appreciate that various modificationsand different combinations of disclosed features are possible withoutdeparting from the scope of the invention.

The invention claimed is:
 1. A calibrated inspection unit for directapplication to a skin surface of a patient, wherein a skin includeslayers below the skin surface, said calibrated inspection unitcomprising: an optical array of elements in an optical path, saidoptical path comprising: at least one light source, an image capturingdevice configured to capture images and having a field of view, areference viewing surface through which the image capturing device isable to capture images, an imaging lens having a radius of curvature anda focal length, wherein the imaging lens is configured to define aposition of a focus point of the image capturing device that extendsbeyond the reference viewing surface along the optical path, a fixedfirst calibration pattern located in a fixed position with respect tothe reference viewing surface and located external to the referenceviewing surface for contacting the skin surface, the fixed firstcalibration pattern being located at the fixed position that is at adepth that corresponds to at least one layer of the layers of the skinbelow the skin surface, wherein the image capturing device is configuredto capture an image of the fixed first calibration pattern, as a firstimage, and the image capturing device is configured to capture an imageof the skin below the skin surface, as a second image, and a focusingmeans for changing the focus position along the optical path as afunction of adjustment values, the calibrated inspection unit furthercomprising: a calibration defining a relationship between the adjustmentvalues of the focusing means and the positions of focus points below theskin surface, the adjustment values including a first adjustment valuefor the position of the focus point of the image capturing device at thefixed position of the fixed first calibration pattern, wherein thefocusing means are provided by the imaging lens which is a liquid lens,and wherein the calibrated inspection unit is configured in a way suchthat the fixed first calibration pattern is configured to be used forfocus calibration.
 2. The calibrated inspection unit according to claim1, wherein said at least one light source is centered on a firstwavelength and has a first spectral bandwidth.
 3. The calibratedinspection unit according to claim 2, further comprising a plurality oflight sources, each light source being centered on a differentwavelength and having a spectral bandwidth, and wherein first adjustmentvalues of the focusing means and the positions of the focus points alongthe optical path are different for each wavelength or for differentwavelengths.
 4. The calibrated inspection unit according to claim 1,further comprising a second calibration pattern located in a secondfixed position with respect to said reference viewing surface and in thefield of view of the image capturing device, and a stored thirdadjustment value for a second focus position at the second fixedposition of the second calibration pattern.
 5. The calibrated inspectionunit according to claim 1, further comprising a front plate in an exitpupil of the optical array, and wherein the fixed first calibrationpattern is provided on at least one of the two surfaces of the frontplate and/or inside the front plate.
 6. The calibrated inspection unitaccording to claim 5, wherein a second front plate is provided at asecond exit pupil and at least one further calibration pattern isprovided on the second front plate.
 7. The calibrated inspection unitaccording to claim 6, wherein the second optical array comprises a lenswhose focal length relates to the length of the second optical array. 8.The calibrated inspection unit according to claim 1, wherein thefocusing means is configured to be translated along an optical axis ofthe optical path or the focusing means is provided by the imaging lenswhich is the liquid lens.
 9. The calibrated inspection unit according toclaim 1, wherein the first and second adjustments values are drivingvoltages.
 10. The calibrated inspection unit according to claim 1,wherein the fixed first calibration pattern is a three-dimensionalpattern defining a plurality of fixed positions from the referenceviewing surface wherein when said three-dimensional pattern isinstallable at an exit pupil, such that the focusing means furthercomprise a plurality of the adjustment values for a focus position atthe plurality of fixed positions of the three-dimensional calibrationpattern.
 11. The calibrated inspection unit according to claim 1,wherein the fixed first calibration pattern comprises a pattern of lineswhich are in a plane which forms an angle with the plane of the imagecapturing device.
 12. The calibrated inspection unit according to claim11, wherein the fixed first calibration pattern comprises parallel oflines which are positioned in a plane parallel to the plane of the imagecapturing device.
 13. The calibrated inspection unit according to claim12, wherein distances between the parallel lines are correlated to theirdepth within a substrate.
 14. The calibrated inspection unit accordingto claim 13, further providing a pre-determined potentially non-lineartransformation for correcting a predefined or pre-calibratedrelationship of a focus position along the optical path as a function ofadjustment values.
 15. The calibrated inspection unit according to claim11, wherein the fixed first calibration pattern is a phantom of humanskin.
 16. The calibrated inspection unit according to claim 1, furtherconfigured to operate with a second optical array providing a secondexit pupil.
 17. The calibrated inspection unit according to claim 1,wherein calibration information is put in optical codes next to thefixed first calibration pattern.
 18. The calibrated inspection unitaccording to claim 1, wherein the fixed first calibration pattern alsocomprises a color calibration.
 19. The calibrated inspection unitaccording to claim 1, further comprising a means to detect when theviewing surface touches the skin.
 20. The calibrated inspection unitaccording to claim 1, wherein the focusing means comprise an imageprocessor configured for calculating a modulation transfer function ofthe optical array.
 21. A method for calibrating an inspection unit fordirect application to a skin surface of a patient, wherein a skinincludes layers below the skin surface, said inspection unit comprisingan optical array of elements comprising at least one light source havinga first spectral bandwidth centred on a first wavelength, an imagecapturing device configured to capture images and having a field ofview, a reference viewing surface through which the image capturingdevice is able to capture images, an imaging lens having a radius ofcurvature and a focal length, wherein the imaging lens is configured todefine a position of a focus point of the image capturing device thatextends beyond the reference viewing surface along an optical axis atsaid first wavelength, and a reference viewing surface, a focusing meansfor changing the focus position along the optical axis as a function ofdriving values, wherein the method comprises the steps of: providing afixed first calibration pattern located at a fixed position with respectto said reference viewing surface and located external to the referenceviewing surface for contacting the skin surface and in the field of viewof the image capturing device, the fixed first calibration pattern beinglocated at the fixed position that is at a depth that corresponds to atleast one layer of the layers of the skin below the skin surface,capturing by the image capturing device an image of the fixed firstcalibration pattern, as a first image, and capturing by the imagecapturing device an image of the skin below the skin surface, as asecond image, establishing a relationship between adjustments of thefocusing means and the positions of the focus points below the skinsurface, the adjustments including a first adjustment value for theposition of the focus point of the image capturing device at the fixedposition of the fixed first calibration pattern, wherein the focusingmeans are provided by the imaging lens which is a liquid lens, andwherein the fixed first calibration pattern is configured to be used forfocus calibration.